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Abstract 

This paper describes the implementation of a quantitative cost effectiveness analyzer for Web-supported 
academic instruction that was developed in Tel Aviv University during a long term study. The paper 
presents the cost effectiveness analysis of Tel Aviv University campus. Cost and benefit of 3,453 courses 
were analyzed, exemplifying campus-wide analysis. These courses represent large-scale Web-supported 
academic instruction processes throughout the campus. The findings were described, referring to students, 
instructors and university from both the economical and educational perspectives. The cost effectiveness 
values resulting from the calculations were summarized in four "coins" (efficiency coins=$; quality coins; 
affective coins; and knowledge management coins) for each of the three actors (students, instructors and 
university). In order to examine the distribution of those values throughout the campus assessment scales 
were created on the basis of descriptive statistics. The described analyzer can be implemented in other 
institutions very easily and almost automatically. This enables us to quantify the costs and benefits of 
Web-supported instruction on both the single-course and the campus-wide levels. 
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Introduction 


Online learning is being widely adopted in higher education institutions around the globe (Allen & Seaman, 
2008; Bonk & Graham, 2006; Mioduser & Nachmias, 2002; Moore & Kearsely, 2005). More and more 
educational institutions are making large investments in instructional technologies on the assumption that 
technology will somehow lead to improved educational quality and eventually to reduced costs through 
greater efficiency (Moonen, 2000). However, the various factors and considerations that go into their 
decision making - whether on the long-term policy level, or that of the distance and online learning 
environments operator i.e., the instructor - are often left unrecognized. If we consider the wide variety of 
distance and online learning models and strategies implemented in diverse settings, from Web-supported- 
academic instruction through blended learning up to a fully on-line model (Bramble & Panda, 2008), the 
mission is even more difficult. 

The institution policymakersneed to be able to evaluate the cost effectiveness of online learning and 
instruction in order to make informed decisions about the extent to which this new technology should be 
used in their institution. Traditional cost effectiveness approaches are inefficient when it comes to 
examining the implementation of technology in teaching (Moonen, 2003, 2005), and they do not suit the 
new needs arising from the rapid pace of Internet implementation in academic instruction. Moreover most 
research that measured the effectiveness of online learning includes assessment of a single course or a few 
courses, but does not offer a campus wide assessment. 

Most studies focused on cost effectiveness of online learning in comparison with traditional face-to-face 
instruction (Russel, 2000; Rumble, 2001: Zhang, 2005; Lee & Chang, 2008). Furthermore, effectiveness 
is measured according to the traditional "class" model, which does not always reflect new learning practices 
that have emerged as a result of exposure to innovative technologies (Bates& Poole, 2003; Gaytan & 
McEwen, 2007). Most research measures the effectiveness of online learning by examining student 
outcomes, such as grades and test scores, student/instructor attitudes about online learning, and overall 
student satisfaction (Alexander & Mckenzie, 1998; Stewart et al., 2004; Kelly et al., 2007). These studies 
measure only some aspects of course effectiveness (Nachmias, 2002). One of the unique tools for 
evaluating learning processes in course websites is analysis of the computer log generated when accessing 
these sites. Evaluation of online courses by analysis of computer logs does not suffer from bias due to 
self-report methods. In addition, information accumulates automatically, and it is stored in digital format 
which is easily accessed for future processing and analysis. Monitoring the students' learning is an 
essential component of high quality education (Mazza & Dimitrova, 2004). 

In light of these issues, a comprehensive cost effectiveness model of Web-supported academic instruction 
was developed, validated and implemented at Tel Aviv University (TAU). The model addresses the new 
needs that arise due to the rapid pace of internet implementation in instruction. It was designed primarily 
for assessing Web-supported academic instruction, and it considers the cost effectiveness of blended 
learning (rather than distance learning only). Thus, it resonates with a growing tendency in the 
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universities. It is nurtured by empirical data taken from Web-logs reflecting students' and instructors' 
usage and provides quantitative analysis of the pedagogical and economical benefits of Internet use for 
students, instructors and the university. The model enables to quantify the cost and benefit on both the 
single course level and the campus-wide level. Consequently the use of the analyzer is twofold: to provide 
comparative results for the instructors in the single course level (for more details, see Cohen & Nachmias, 
in press), as well as to provide a campus wide analysis by summarising the cost and benefit of all courses 
throughout the campus as this paper illustrates. This model can help the managers of educational 
institutions or the faculties to support their decision making, and more generally it may stimulate a 
consciousness-raising process about investments in the field of information and communication 
technology (ICT) in education. 

In this paper we will first briefly describe the cost effectiveness model and its computational analyzer. 
Subsequently, we will present its implementation at the Tel Aviv University campus. Cost and benefit of 
3,453 courses provided by TelAvivUniversity were analyzed, exemplifying campus-wide analysis with the 
model. These courses represent large-scale Web-supported academic instruction processes throughout the 
campus. The findings will be described, referring to students, instructors and university from both the 
economical and educational perspectives. The main objective of this paper is to demonstrate the model's 
potential in estimating the cost effectiveness of large-scale Web-supported instruction processes in a 
university, with special reference to the main actors involved. 

The Cost Effectiveness Model 

The cost effectiveness model for Web-supported academic instruction presented in this paper is based on 
the insight that there is no one main benefit from integrating the Internet in education but many benefits 
in different dimensions. The model consists of: (a) a cost effectiveness framework that defines cost and 
benefit components of Web-supported academic instruction (for full details, see Cohen & Nachmias, 
2006), and (b) a computational analyzer which translates the cost effectiveness components into 
quantitative values (Cohen & Nachmias, 2007). To this purpose a different measure for each of the 
components was developed, enabling its calculation and quantification in relation to each of the three main 
actors involved in the learning instruction process: students, instructors, and University. Model validation 
was conducted by reference to theoretical and experimental literature, experts in Web-based academic 
instruction and instructors using Web-supported instruction. This section will first describe the cost and 
benefit components of the model, then the computational analyzer. 

Cost effectiveness framework 

The framework includes 44 benefit components and 23 cost components in the following six dimensions: 

1. Increasing efficiency of teaching and learning processes - One of the main benefits of Internet 
implementation is reaching optimal balance between reducing costs to a minimum, e.g., reducing 
time, effort and costs, and increasing results to maximum efficiency (Bishop, 2006; Bonk & 
Graham, 2006; Lewis & Orton, 2006; Twigg, 2003). This dimension includes cost reduction and 
time saved as a result of implementing the Internet, e.g., reducing time and money due to learning 
flexibility and new mode of interaction; reducing time and money in contents consumption, in 
searching and finding them; reducing instruction time, since students spend ample and efficient 
time in learning from contents and peers; saving classroom costs, lecturer hall or lab costs; saving 
lab equipment costs by increasing equipment use or saving in consumed equipment. 

2. Improving instruction quality - This dimension refers to improving the effectiveness of pedagogical 
aspects by enriching the learning environment. Using synchronic and asynchronic innovative 
learning activities in which students experience new types of learning in active and dynamic 
learning environments; using constructivist activities that lead to flexibility, active learning, 
collaboration and enhanced productivity (Collis & Moonen, 2001; Zhang, 2005; Kim, Liu & Bonk, 
2005). Exposure to varied types of contents; relevant and updated visual and digital instructional 
materials; exposure to illustration enabled by the interactive system, e.g. simulations or interactive 
software, assisting the student in understanding complex processes via system feedback; 
synchronic and asynchronic interpersonal communication between instructors and students, 
among students (Wu & Hiltz, 2004)and with experts all over the world; sharing of information and 
ideas between users and universities using the Internet for displaying contents and products such 
as student papers; means of assessment and testing, such as online exams, self-testing, tasks and 
projects; and feedback on instruction activities. This dimension is related to achievement of 
instructional goals: improving grades and understanding, acquisition of learning and technological 
skills, development of cognitive skills, and acquisition of life-long-learning skills. 

3. Improving affective aspects - This dimension includes increasing students' and instructors' 
motivation, interest, self-confidence with reference to technology, attitude and satisfaction from 
the course, from the change in teaching and learning methods, from the user-friendliness of the 
system; and the prestige of the "wired" institute. Affective aspects of the instructional and learning 
processes are usually considered as added values but there is no doubt that a vital aspect of any 
education is to ensure that students' and instructors' affective needs and characteristics are met. 
Researchers (Bonk & Graham, 2006; Bourne & Moore, 2004; Zaharias et al., 2004; Belanger & 
Jordan, 2000) found that the degree of motivation to learn and its main determinants Attention, 
Relevance, Confidence and Satisfaction, can be contributed by a set of e-learning usability 
attributes (such as navigation, learnability, visual design, accessibility, consistency) and 
instructional design attributes (such as instructional assessment and feedback, learner guidance 
and support, learning strategies design, interactivity/engagement etc.). 

4. Knowledge management improvement - Use of a management system allows effective knowledge 
organization as well as greater collaboration, information exchange, and sharing of resources and 
instructional materials (e.g., using the course Website over the years, or sharing it with other 
instructorswithin and between institutionsin the country and abroad, creating multi-databases 
including resources from various disciplines and faculties, and creating new courses from existing 
instruction units) (Greenberg, 2002; Sitzmann et al., 2006; Vilaseca & Castillo, 2008). 

5. Infrastructure costs - This dimension refers to technological infrastructure costs, such as central 
infrastructure and equipment costs (e.g., servers, software and communication), and operational 
infrastructure costs, i.e., institute support centre, training, workshops, as well as continuous 
technological and pedagogical support (i.e., preliminaiy and ongoing support for faculty and 
students, and implementation costs). 

6. Instruction costs - Course development and preparation, costs of curriculum development and 
course production; these costs are reflected in the amount of instructional materials embedded in 
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the Website (Rumble, 2001b), and they also include assessment time and interaction time with 
students. Instruction/ learning costs are usually dependent on the number of registered students 
and fixed costs of course development and delivery. Faculty time affects (i.e., enlarges) cost of 
online course production (Bacsich et al., 1999; Oliver et al., 1999). The highest cost in an online 
course was found to be faculty time, due to the nature of the course using active online discussion 
(Jung, 2005). These costs can be reduced if the course design is less interactive (Bartolic-Zlomislic 
& Brett, 1999). 


Figure 1. A computational analyzer 
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A computational analyzer 

A computational analyzer was developed in order to translate the 67 cost and benefit components of the 
model into quantitative values. For each one of the components, computational functions (Y=f(X)*M) are 
defined. These functions calculate quantitative values for each of the three main actors involved in the 
learning process: students, instructors, and the academic institution's policymakers. The indicators 
(X{xi...x93}) are extracted from the sites' Web-logs. The indicators are independent variables that 
characterize the course, the course Website, the Web-based teaching processes and their use by students. 
The cost effectiveness parameters (M={mi...m82}) translate the costs and benefits derived from the 
independent variables into a quantitative measures in terms of four different "coins" on a cost effectiveness 
scale .The computational mechanism is the collection of all functions. The indicators (X) and parameters 
(M) constitute the input of this mechanism; the computational mechanism (through the functions) 
processes the data to produce the desired output. The output consists of the cost effectiveness values for 
each of the three actors in terms of four different "coins": "efficiency coins" in the form of money ($) and 
time (hours), "quality coins" (Q) for improving instruction and learning quality, "affective coins" (A) for 
improving affective aspects and "knowledge management coins" (KM) received through facilitating 
knowledge management (Figure 1). 

Although the development process is tedious and includes a very large number of definitions (67 
components, 93 indicators, 82 parameters and 108 functions), activating the analyzer is rather simple. The 
indicators and parameters are represented in two spreadsheet input files; all the definitions of the 
functions are represented in a third file; and the fourth file represents the cost effectiveness values yielded 
by the calculations. The amount of coins received in these dimensions indicates the effectiveness level 
found. Anyone who uses the analyzer can insert the input data manually or produce them from the 
Web-supported shell. He or she can also define the parameters for each measurement according to 
case-sensitive predisposition. Then the computational analyzer is activated and the computational 
mechanism processes the data to produce the output. 

Methodology 

The cost effectiveness analyzer was implemented in 3,453 courses provided by the Tel Aviv University in 
the year 2007; these courses represent large-scale Web-supported academic instruction processes 
throughout the campus. During this year 23,352 students were enrolled in these courses and 1,850 
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instructors were teaching them. The aim of this study, only part of which is described in this paper, is to 
estimate the cost effectiveness of large-scale, campus-wide Web-supported instruction processes. 

The cost effectiveness analyzer was automatically implemented in each one of the 3,453 courses. The 
computational mechanism processed the data and produced the courses' output files regarding cost and 
benefit for a period of a year in relation to students, instructors and university. The cost effectiveness 
values resulting from the calculations were summarized in four different "coins" (efficiency coins=$; 
quality coins; affective coins; and knowledge management coins) for each of the three actors. In order to 
examine the distribution of those values throughout the campus assessment scales were created on the 
basis of descriptive statistics using measures of centralization and dispersion. The aim was to demonstrate, 
both graphically and numerically, the distribution of the cost and benefit components in the courses 
campus wide. 

Results 


The cost effectiveness analyzer was first implemented in 3,453 courses run at Tel Aviv University in 
September 2007 in order to estimate, on the campus-wide level, the cost effectiveness of large-scale 
Web-supported instruction processes. 23,352 students were enrolled in these courses which were taught 
by 1,850 instructors from eleven academic units (9 faculties and 2 independent schools). The total cost and 
benefit derived from Web-supported academic instruction was estimated for each of the three actors 
involved: students, instructors, and university, in the six dimensions of the model. 

Table 1 presents a summary of the total cost and benefit analysis for each of the three actors involved: 
students, instructors, and university in six dimensions of the model. 

Table 1. Total Cost and Benefit Analysis Campus-Wide 


Costs and Benefits 
Dimensions 

"Coins" 

Students 

N = 23,352 

Instructors 

N = 1,850 

University 

Infrastructure costs 

$ 

— 

— 

$237,030 

I nstr uction- learning 
costs 

$ 

$59,164 

$1,044,935 

___ 

Efficiency 

$ 

$10,737,285 

$499,521 

$18,514 

Instruction quality 

Quality coins (Q) 

34,914,554 Q 

4,132,413 Q 

— 

Affective aspects 

Affective coins (A) 

26,555,712 A 

2,329,332 A 

28,885,044A 

Knowledge management 

Knowledge Management 
coins (KM) 

121,760 KM 

649,179 KM 

115,428KM 


1$= 3.6 New Israeli Shekel (NIS) 


The data in Table 1 show that all three actors involved in online instruction gain enormous benefits in 
terms of the four "coins" (efficiency, quality, affective aspects and knowledge management) - far beyond 
the cost invested. The main investors are the instructors while the large number of students using the 
learning management system reaps the greatest benefits. Furthermore it was found that the university 
gained prestige at a relatively small cost and the instructors gained the largest knowledge management 
benefits. 

Total direct cost of integrating Websites in academic instruction was measured by means of 21 
computational functions and was found to beabout $1,350,000. About 78% of this sum went into 
instructors' development and implementation of online instruction, including the time invested in 
interaction with students and in assessment (instructor time was converted to money, lhour = $30). The 
university invested about 18% of the costs in the technological and operational infrastructure, and about 
4% was spent by the students, mostly for printing the electronic learning materials. It is not surprising that 
the instructors and the university are the main investors while the students reap most benefits. 

Increased efficiency of teaching and learning processes was measured by means of 28 computational 
functions: About $10,740,000 was saved by students as a result of electronic content consumption 
efficiency, receiving/delivering electronic announcements, performing exercises on-line, posting papers 
and assignments on the Web, and saving copying/printing costs. About $500,000 was saved by instructors 
due to intensive Website usage for delivering instructional contents, exercises, administrative information 
to students and Website use as communication channel with time and space flexibility. About 95% of the 
total saving was on behalf of the students, the lion's share of which derived from saving time that was 
converted to money. The university saved only around $20,000 since most of the courses were Web 
supported, i.e., internet integration was aimed to enrich teaching-learning processes on campus and not to 
replace them. Factors such as saving on physical infrastructure cost and manpower were not realized. 

Improved instruction quality was measured by means of 24 computational functions. "Quality coins" (Q) 
were calculated for various pedagogical activities (e.g., one viewing of on-line paper = 2Q; activating a 
simulation = 5Q; viewing lesson recordings = 4Q; one forum posting = lQ, etc.). It was found that 
instruction and learning quality were significantly improved. In the year 2007 about 35,ooo,oooQ were 
awarded for student learning as a result of activity based on interpersonal communication, various content 
knowledge representations, and in particular for self-exercise accompanied by immediate feedback that 
enables individual learning at a personal pace. The fact that the instructor received over 4,000,000 "Quality 
coins" (e.g., one posting of reply message to student in forum = 0.5Q), suggested improved instruction as a 
result of interpersonal communication, intensive Website use, online assessments processes that yield 
activity reports, which in turn made it possible to supervise student learning. The university did not benefit 
directly from the quality improvement yet through the resulting gain in prestige it benefited under the 
affective component. 

Improved affective aspects, such as satisfaction, were measured through 15 computational functions. 
Students received a very high value of "Affective coins" (about 27,ooo,oooA), reflecting satisfaction as a 
result of simplicity of use; interactivity; immediate feedback; flexible learning; interaction between lecturer 
and students. The instructors were satisfied with the students' Website usage, with the convenience of 
flexible instruction, and with the increased interaction with their students (about 2,30o,oooA). The 
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university scored a very high affective benefit (29,ooo,oooA) for the prestige gain that was the result of 
students' and instructors' satisfaction and instruction quality improvement. 

Knowledge management improvement was measured through five computational functions. The students 
and the university received Knowledge Management benefits from working with the course Websites 
under one Learning Management System (about 120,000 KM "Knowledge Management coins" for 
students and about 115,000 KM for the university). But the great winners were the instructors, who 
received about 650,000 KM for effective knowledge organization (e.g., greater collaboration and sharing 
among instructors, and reusing materials over the years). 

Tables 2-3 present the distributions of the students' and the instructors' cost and benefit measures, whose 
sum total was displayed in Table 1. These tables present both numerically and graphically, descriptive 
statistics regarding the cost and benefit measures of all 3,453 courses. In the graphic representations the 
maximum values do not appear for the sake of providing a clearer scale of the other measures. 

Table 2 presents descriptive measures for students' cost and benefit in 3,453 courses. The data reveal that 
none of the distributions were normal, and the high standard deviations indicate high variance among 
courses. If we would take a further look at the courses distributions which are not detailed here, we could 
see that it is the case in all dimensions that only a few courses gain the lion's share of the students' benefits. 
For example, in the efficiency dimension 10% of the courses were responsible for 50% of the total saving. 
In 5% of the courses between $11,000 and $28,000 was saved by students per course, and in only 1 % of 
the courses (35 courses) in excess of $28,000 was saved by students per course (this large saving was a 
combination of the large number of students in the relevant courses and instructor's great investment). 
Another example for large course variance is in the quality improvement dimension: 90% of the students' 
quality benefits were gained in 25% of the courses only. In 50% of the courses low quality benefits were 
found for the students. 



Table 3 presents the descriptive measures for instructors' cost and benefit regarding the 3,453 courses. 
From this table, like in the case of the students, we can see the not normal distributions of the courses in 
terms of instructors' cost and benefit, and the high variance among courses. Here again it is a relatively 
small number of instructors who gain the majority of the benefits. Only 5% to 10% of the course instructors 
gained most of the benefits in each dimension. In the quality dimension for example, instructors benefit 
from integrating the web only in 20% of the courses. In 80% of the courses no quality benefit was found at 
all. However in the affective dimension instructors were rather satisfied with internet introduction. Even 
though some instructors received a negative value, indicating lack of satisfaction, most of them were 
satisfied. Few instructors, moreover, scored extremely high values in this dimension, when the maximum 
value was 7,901 (the average was 675). 

Table 3. Descriptive measures of Instructors' Cost and Benefit (N = 3,453 courses) 



Sum 

* 

average 

standard deviation 

mode 

1 

MIN 

1 

quartilei 

♦ 

median 

1 

quartile 

3 

MAX 


Learning 

cost 

$1,044,935 

303 

636 

5 

2 

51 

116 

258 

14,255 


- 1 

,- 1 1 - 

1 

t - ' 

) 100 200 3 
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$499,521 

145 

823 

2 

0 

3 

11 
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Discussion 


The cost effectiveness model presented in this paper demonstrates that the quantification of the cost and of 
the many benefit components of Web-supported instruction is possible. It uses empirical data taken from 
students' and instructors' Web-logs and provides quantitative analysis of the pedagogical and economical 
benefits of Internet use on the campus-wide level, referring to students, instructors and the university as 
such. Since it is nurtured by Web-logs and Web-mining techniques are used, the analyzer is very easy to 
operate and can be applied almost automatically to a large number of cases. Any educational institution 
that integrates the internet in instruction can easily implement it. 

The contribution of this study, only part of which is described in this paper, is on two levels: On the 
theoretical level, this study focuses on extending the body of knowledge on Web-supported academic 
instruction cost effectiveness. The study provides a framework for comprehending whole components 
taken into account when analyzing the costs and benefits of Web-based instruction and learning. The 
model provides quantitative analysis of the pedagogical benefits resulting from Internet use. Thus it meets 
the challenge of analyzing benefits resulting from the improvement of learning-instruction processes, not 
only from economic factors. On the practical level - the calculations that translate the cost effectiveness 
components of Web-supported academic instruction into quantitative values are easy to use and can be 
performed almost automatically. Therefore the model is applicable in any academic institute implementing 
Web-supported instruction. 

From the campus-wide analysis we can draw two main preliminary and apparently contrasting conclusions. 
The first is related to the rapid diffusion of internet use and the impressive benefits achieved at Tel Aviv 
University as a result of integrating the internet in the academic instruction. And this has been done at 
relatively little cost. The main investors are the instructors while the large number of students using the 
learning management system reaps the greatest benefits at the lowest cost. The university, at relatively low 
cost in technological and operational infrastructure, gained prestige and improved its instructors' 
knowledge management. The second conclusion relates to the fact that the web's potential for increasing 
efficiency, improving quality and effective knowledge management was only exploited up to a point, at the 
campus (Zhang, 2005; Garrison & Kanuka, 2008). The variance among the courses was found to be very 
high. Moreover the fact that in most of the courses benefits were rather low indicates that a relatively small 
number of courses gain large benefits from integrating the internet. Nevertheless it seems that Web usage 
in the teaching learning processes does yield instructor and student satisfaction. 

The cost effectiveness analyzer tool enables the evaluation of costs and benefits of different types and 
modes of Web-supported courses, as well as to compare faculties, degrees, and different kinds of courses. 
The analyzer can also measure a course's cost and benefit in a certain year or compare them over the years 
to get a picture of a process rather than a current state snapshot. This paper presents the total cost 
effectiveness analysis of all the courses on campus that were using the internet. In future research, subsets 
of courses can be analyzed in order to evaluate cost and benefit of different types and modes of 
Web-supported courses, to compare faculties, degrees, and different kinds of courses etc. The analyzer can 
also measure the course cost and benefit in one year or compare cost and benefit in several years to get a 
cost and benefit picture of a process rather than a current state snapshot. The measures can be per student, 
per course, per faculty or any other parameter. The model can serve as basis for simplified return on 
investment analysis (Moonen, 2003) and to support decision makers. 

The analyzer enables us to calculate the cost and benefit of integrating Internet technologies for three main 
actors: the university, the instructors, and the students - in a certain course (see Cohen & Nachmias, 
2008), in a faculty and in campus-wide as this paper illustrates. These calculations can support decision 
making possesses and can serve as a basis for return on investment analysis. Consequently the use of the 
analyzer is twofold: to serve as a reflective tool for instructors assessing the online instruction and learning 
(on the distance and online learning systems operator's level) as well as to provide a campus-wide analysis 
to the institution's decision makers (on the long-term policy level). 

As Web-based academic programs are a new experience in most higher education institutions, it is of great 
importance, like in all new endeavors, to learn by reflection on what has already been done, to observe 
challenges and successes, and to gain insights on assumptions and actions. A cost effectiveness model of 
advanced technology implementation in academic instruction can be considered a central tool in this 
reflective examination. The model's components are universal and can be adjusted to the characteristics 
and needs of academic institutions implementing Web-supported instruction. We envision this cost 
effectiveness model as a reflective tool for assessing the emerging trend of Web-supported academic 
instruction and blended learning and for providing a basis for collaboration. Therefore the analyzer will be 
offered upon request. 
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